System and method for performing a computerized simulation of a medical procedure

ABSTRACT

Embodiments of the invention are directed to a method of performing computerized simulations of image-guided procedures. The method may comprise receiving medical image data of a specific patient. A patient-specific digital image-based model of an anatomical structure of the specific patient may be generated based on the medical image data. A computerized simulation of an image-guided procedure may be performed using the digital image-based model. Medical image data, the image-based model and a simulated medical tool model may be simultaneously displayed.

BACKGROUND OF THE INVENTION

Many invasive medical procedures such as for example endovascularprocedures can pose challenges even to the most experiencedinterventionists. Tortuous anatomy, difficult visualization, complexlesion morphology, and other complications can add to increasedprocedure time, fluoroscopy exposure, and contrast dye use. Precioustime can be lost if the access strategy or equipment choice issuboptimal. Accordingly, simulation systems for image-guided proceduresfor training a physician without unnecessary risk, which may serve aspre-operative planning tool or post-operative assessment tool, have beenintroduced.

Most medical simulations may enable a doctor to practice on a virtualmodel that is based on predefined models of anatomical structures. Thesestructures are not associated with a specific patient. Accordingly, suchsystems cannot be used for accurately planning an operation on aparticular patient prior to performing the operation or for postassessment of the operation.

A more progressive simulation system is a patient-specific simulationsystem that uses patient-specific medical image data. A patient-specificsimulated interventional environment may allow rehearsing a completecomplex procedure such as endovascular procedure on a virtual model of apatient's exact anatomy, increasing the likelihood for an effectiveoutcome in real life.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter regarded as the invention is particularly pointed outand distinctly claimed in the concluding portion of the specification.The invention, however, both as to organization and method of operation,together with objects, features and advantages thereof, may best beunderstood by reference to the following detailed description when readwith the accompanied drawings in which:

FIG. 1 shows an exemplary system for simulating an image-guidedprocedure according to embodiments of the invention;

FIG. 2 is a flowchart diagram illustrating a method for concurrentlysimulating an image-guided procedure and presenting medical image dataaccording to some embodiments of the present invention;

FIGS. 3, 4, 5 and 6 show exemplary screenshots related to an integratedpresentation of a simulated procedure and medical image data accordingto embodiments of the invention; and

FIGS. 7 and 8 show exemplary rendering modes that may be used forpresentation of medical image data according to embodiments of theinvention.

It will be appreciated that for simplicity and clarity of illustration,elements shown in the figures have not necessarily been drawn to scale.For example, the dimensions of some of the elements may be exaggeratedrelative to other elements for clarity. Further, where consideredappropriate, reference numerals may be repeated among the figures toindicate corresponding or analogous elements.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

In the following detailed description, numerous specific details are setforth in order to provide a thorough understanding of the invention.However, it will be understood by those having ordinary skill in the artthat the invention may be practiced without these specific details. Inother instances, well-known methods, procedures, components, modules,units and/or circuits have not been described in detail so as not toobscure the invention.

Embodiments of the invention are directed to patient-specificcomputerized simulations for image-guided procedures. In someembodiments, a method for enabling a performance of a simulatedprocedure may include producing a digital image-based model of ananatomical structure based on medical image data received from a scan ofa subject. The subject may be, for example a patient that is about toundergo an image-guided procedure. According to embodiments of theinvention, the medical images or other data may be patient-specificmedical images and parameters obtained from one or more imaging systemssuch as computed tomography (CT), a CT-fluoroscopy, fluoroscopy,magnetic resonance imaging (MRI), Ultrasound, positron emissiontomography (PET) and X-Ray systems or any other suitable imagingsystems. Embodiments of the invention may use as input medical imagedata described herein to produce a 2D, 3D or 4D model of an anatomicalstructure, organ, system or region. Although a simulated anatomicalorgan or system may be intuitive and may accurately depict anatomicalaspects, it may also provide a different view with respect totraditional images or presentations the medical personnel is familiarwith and/or accustomed to. For example, a physician may typically beused to examining X-Ray or CT images and may be well trained to identifyanatomical structures or organs in such images but may find it difficultto identify or determine locations or orientations of organs or toolswith respect to a simulated model.

According to embodiments of the invention, a method of performing acomputerized simulation of an image-guided procedure may comprisereceiving medical image data that may be related to a specific patient,generating a digital, image-based, model of an anatomical structurebased on the received medical image data, conducting the actualsimulated image-guided procedure and simultaneously displaying thedigital model and the medical image data. For example, a first region ina display screen may be dedicated to displaying a simulated model of ananatomical organ while a second region on the same or other display maydisplay a corresponding CT image of the simulated organ. In someembodiments, the medical image data, e.g., a CT slice image may bedisplayed in overlay mode on the simulated model. For example, a CTslice image may be displayed in the same display region the simulatedmodel is displayed and may further be positioned according to variousorientations or view selections.

As described herein, embodiments of the invention may provide a userwith various views of images and other medical data. For example, viewsprovided may be related to a predefined surface or plane. In someembodiments, an image related to a specific surface may be generatedbased on received raw medical image data, artificially generated medicalimage data or a combination of raw and artificial medical image data.For example, CT slices or other images may be processed to produce viewsthat may not necessarily be related to the view of the imaging systemused for acquiring the CT data. In some embodiments, views generated maybe related to specific planes, surfaces, angles or other aspects asneeded. Some exemplary planes or surfaces for which views may begenerated may be an axial plane, a coronal plane, a sagittal plane, aplane orthogonal to a selected vessel, a non-planar surface, a planerelated to an imaging system used to acquire said medical image data, ora plane related to a virtual imaging system, which forms part of thesimulated procedure. It will be noted that while some exemplary plane orsurface views are described herein, any such views may be generated asdescribed herein. It will further be noted that the terms “surface” and“plane” may refer to substantially the same entity or aspect of a viewwhen in applicable context, and may accordingly be used interchangeablyherein. For example, a sagittal plane view and sagittal surface view mayrefer to the same view.

In some embodiments, an image related to a specific surface, point ofview or imaging system may be generated based on received raw medicalimage data. For example, CT slices or data may be processed to produceviews that may not necessarily be related to the view of the CT system.For example, a two dimensional X-Ray view may be produced based on adigital image-based model. In a particular scenario, a 3D digitalimage-based model may be generated based on input from a CT system asdescribed herein. An X-Ray view may then be generated based on thedigital image-based model. For example, a two dimensional (2D) X-Rayview may be produced by projecting a three dimensional digitalimage-based model on a two dimensional surface or plane. Embodiments ofthe invention may enable a user to select an X-Ray camera view, positionor location with respect to a patient and produce an X-Ray image or viewthat would have been provided by an X-Ray camera located at the selectedlocation and/or position. Accordingly, artificial or synthesized X-Rayimages or views may be generated and provided even when no actual X-Raycamera is utilized for obtaining such images. In another embodiment,data received from a CT system may be used to generate a digitalimage-based 3D model and such model may further be used to generate aview that would have been provided by a camera fitted on a colonoscopeused for examination of the colon. Accordingly, views revealing internalaspects or views of the colon may be generated and displayed. Likewise,views or images related to any applicable technology or system may beproduced based on a digital image-based model. The ability to generateviews or images normally produced by various systems may be highlyappreciated by those of ordinary skill in the art. Advantages of suchability may include a substantial saving of cost and time since based ondata received from a first imaging system, embodiments of the inventionmay provide output related to a second, different system. Anotheradvantage may be the ability to synthesize and provide images related toany angle, orientation, position or location of a simulated imagingdevice. Artificial views related to virtual systems as described hereinmay be displayed, e.g., in addition to, or instead of, the image-basedmodel described here.

As described herein, displaying of medical image data such as CT, MRI orX-Ray images may be integrated or coordinated with, or otherwise relatedto a presentation, a progress or other aspects of a simulated procedure.In some embodiments, presentation of medical image data may be accordingto a selection of a user. For example, a user may click a location on asimulated model, or otherwise select or indicate an interest in alocation, anatomical organ or region related to the simulated model andbe provided with a CT or other medical image corresponding to theselection. In other embodiments, presentation of medical image data maybe automatically synchronized or otherwise coordinated with a progress,state, mode or context of a simulated procedure. For example, CT imagesrelated to a location of a catheter, a wire, a stent or anothersimulated tool that may be shown in a simulated procedure may beautomatically displayed such that a CT image related to an area, aregion or a volume (of a real patient, as imaged by a CT imaging system)where the stent is located (in the simulated model) may be shown. As thetool is moved from a first location to a second location, a first CTimage (related to the first location) may be automatically replaced by asecond CT image (related to the second location) such that a CT image ofthe second location is presented instead of a CT image of the firstlocation. Accordingly, a presentation of medical image data may beautomatically synchronized, matched and/or coordinated with a simulatedprocedure. In addition, scale or other operations as known in the artmay be performed on either a region displaying the medical image dataand the simulated model. For example, a user may zoom in on a regionwithin the medical image data while leaving the display of the simulatedmodel unchanged or alter the scale or size of the region displaying thesimulated model while leaving the scale of the region displaying themedical image data. In embodiments where two separate windows or displayregions are used to present the medical image data and the simulatedmodel, any manipulations of such windows or regions as known in the artmay be enabled.

Reference is made to FIG. 1 showing an exemplary system 100 forsimulating an image-guided procedure according to embodiments of theinvention. As shown, system 100 may comprise a computing devicecomprising a number of units and operatively connected to input and/oroutput (I/O) units. System 100 may include input units 105, output units120, a model generation unit 110, a simulation unit 115, a managementunit 135, a data repository 140, a presentation unit 125, a memory 130and a controller 131. Input units 105 may include a mouse, a keyboard, atouch screen or pad or any suitable input devices. Input units 105 maycomprise means for operating simulation tools. For example, physicalobjects such as handles, activation buttons and the like, as well asreal medical tools may be connected to input units 105 to enable a userto operate simulated tools such as a simulated catheter. Additionally,input units 105 may include a wired or wireless network interface card(NIC) that may receive data, for example, from an imaging system.According to some embodiments, a mediator unit, e.g., a communicationmanagement unit may utilize a NIC to communicate with a system or serverstoring medical images such as a picture archiving communication system(PACS), may obtain any relevant imaging information, data or parametersfrom such a system, server or application and may store obtained data,information or parameters in local data repository 140. Output units 120may include display screens, components for interfacing with a displayscreen to enable visual output or optionally a speaker or another audiodevice to enable audible output. Output units 120 may include one ormore displays, speakers and/or any other suitable output devices. Outputunits 120 may additionally include force feedback components that mayapply, cause or generate physical forces or resistance (e.g., frictionlike resistance) to physical input devices that may be operated ormanipulated by a user. Output units 120 and input units 105 maycommunicate with any other component or units of system 100 andaccordingly may enable such units to communicate with external systems.Units 105, 110, 115, 125 and 135 may be or may comprise software,hardware, firmware or any suitable combination thereof.

Also shown in FIG. 1 are an external image data repository 145 and animaging unit 146. External image repository 145 may be any suitable orapplicable database, repository or archive. For example, external imagedata repository 145 may be a picture archiving and communication systems(PACS) archive or repository. As known in the art, a PACS system maycomprise computing and/or other devices that may be dedicated to thestorage, retrieval, distribution and presentation of medical images.Images in a PACS system may be stored according to various formats,e.g., digital imaging and communications (DICOM). A PACS systemtypically comprises or is operatively connected to an archiving systemfor archiving data such as CT, MRI or other images and related data, asecured network for the transmission of patient sensitive or privateinformation and possibly, computing devices to receive image or otherdata from the archiving system. Embodiments of the invention may beconfigured to interact with a PACS system, e.g., over a networkinterface card (NIC) connected to a network such that communication witha PACS system is enabled. Management unit 135 may interact, e.g., over anetwork and possibly according to and/or by implementing a predefinedprotocol, with external data repository 145 that may be a PACS system.CT, MRI or other images and related data may be thus retrieved, receivedor otherwise obtained from such PACS or other system and may further beused as described herein and/or stored or buffered, for example, in datarepository 140. Imaging unit 146 may be an imaging system such ascomputed tomography (CT), a CT-fluoroscopy, fluoroscopy, magneticresonance imaging (MRI), Ultrasound, positron emission tomography (PET)and X-Ray systems or any other suitable imaging system. As shown,medical image data and/or related data may be received by external imagedata repository 145 from imaging unit 146. External repository 145 mayreceive medical imaging data from any other applicable source, e.g.,over a network from a remote site or hospital or by means of a removablestorage device that may be connected to repository 145. As shown, system100 may receive image data and/or related data directly from imagingunit 146. Although not shown, system 100 may receive imaging and/orother data from any applicable source, accordingly, embodiments of theinvention are not limited by the type, nature or other aspects of asource of medical imaging or other data nor by the way such data iscommunicated and/or received.

Model generation unit 110 may include components or modules forgenerating a digital model and its graphical representation, e.g., a 3Danatomical model of an anatomical structure, such as an organ vesselsystem or any other area of interest within a body of a subject. Themodel may be generated by model generation unit 110 according toinformation received from an imaging system, for example, a medicalimage received from a CT system via input unit 105. It will berecognized that embodiments of the invention are not limited by themethod or system for generating a digital image-based model of ananatomical structure, any methods or systems may be used for generatingsuch model without departing from the scope of the invention. Simulationunit 115 may include components for generating a simulation of animage-guided procedure. For example, when a user performs a simulation,for example as a pre-procedure for an image-guided procedure, usingsimulation unit 115, a graphical representation of a digital model(e.g., produced by model generation unit 110), and the simulationprocess may be displayed on a monitor that may be one of output units120. A generation of a digital model of an anatomical organ, system,section or region (e.g., by model generation unit 110) and a simulationof a procedure (e.g., by simulation unit 115) may be according tomethods, systems and/or other aspects as described in US PatentApplication Publication US 2009/0177454. Controller 131 may be anysuitable controller or processing unit, e.g., a central processing unitprocessor (CPU). Memory 130 may be any suitable memory component,device, chip or system and may store applications or other executablecodes that may be executed by controller 131 and/or data, e.g., datathat may be used by applications or programs executed by controller 131.For example, executable code, applications or modules implementing modelgeneration unit 110 and/or simulation unit 115 may be loaded into memory130 and executed by controller 131.

It will be recognized that system 100 as described herein is anexemplary system. According to embodiments of the invention, system 100may be implemented on a single computational device or alternatively, ina distributed configuration, on two or more different computationaldevices. For example, model generation unit 110 may operate on a firstcomputational device and managed by a first management unit whereassimulation unit 115 may operate on another computational device andmanaged by a second management unit that communicates with the firstmanagement unit. In another exemplary embodiment, management unit 135may operate on a computational device, model generation unit 110 mayoperate on a second computational device and simulation unit 115 mayoperate on a third computational device.

Presentation unit 125 may control, coordinate or manage a display orpresentation of video, audio or other aspects of a simulated procedureand related medical image data. For example, presentation unit 125 mayreceive data, parameters or other information from a plurality ofsources and incorporate received or obtained data into a presentation toa user. Presentation unit 125 may coordinate, synchronize or otherwiserelate a presentation of information from a plurality of sources withina single presentation. For example, presentation unit 125 may coordinateor synchronize a presentation of medical image data, e.g., CT imageswith a presentation and/or performance of an image guided simulatedprocedure. Alternatively or additionally, presentation unit may cause apresentation of information according to a user selection. For exampleand as further described herein, a view selection from a user may bereceived and a presentation of medical image data may be according tosuch selection. Management unit 135 may interact with any module, unit,application or other applicable entity and may perform coordination,scheduling, arbitration, supervising and/or management of flows,procedures or other aspects as described herein.

Reference is made to FIG. 2, which shows an exemplary flowchartdescribing a method for simulating an image-guided procedure accordingto some embodiments of the invention. The method may include receivingpatient-specific medical image data that may be related to a specificsubject or patient about to be treated (box 210). The medical image datamay be received directly from an imaging or scanning system such as forexample, a CT or MRI scanner or alternatively from an external imagedata repository, such as a picture archiving communication system(PACS). As will be understood to a person skilled in the art, themedical image data may be received from any other applicable source.

According to some embodiments, the received medical data may be storedlocally or internally within a data repository, such as data repository140. According to some embodiments, possibly based on received data,data may be generated. For example, based on a set of three hundred(300) CT images, an extended set of six hundred (600) may be generated.For example, by examining an initial set of images, embodiments of theinvention, e.g., executable code executed by controller 131, maygenerate or produce additional images and thus produce a new set ofimages. For example, if a given set of images was produced by acquiringa single image per a movement of one centimeter (1 cm) of the imagingunit, a new set may be produced by artificially generating images toreflect a movement of half a centimeter (½ cm) of the imaging unit. Forexample, based on two sequential images in an original set, anadditional image may be artificially generated, e.g., by observingvariations between the two original images. Such artificially generatedimage may be inserted between the two original images in order toproduce a new set of images that may be coherent and may adequatelyrepresent the anatomy or other aspects of a related patient. Suchgeneration of images may enable embodiments of the invention to providebetter resolution, and enhance a correlation of medical data and asimulated procedure as described herein. A set of images, either asreceived or extended as described herein may be stored locally, e.g., indata repository 140.

As described herein, raw medical image data, e.g., acquired by, andreceived from an imaging system may be processed. Processing of medicalimage data may be performed prior to generating views or image basedmodels as described herein. For example, pixels for rendering may beproduced by interpolation applied to acquired pixels or other imagerelated data. For example, CT slices may be artificially generated byinterpolation of acquired CT slices. Windowing transformations as knownin the art may be another example of processing that may be applied tomedical image data by embodiments of the invention, e.g., in order toenhance, alter or otherwise modify aspects such as contrast orbrightness of an image that may be received from an imaging system ormay be artificially generated as described herein.

As shown by box 215, the method may include generating a model of ananatomical structure based on medical image data. Generation of ananatomical structure may be based on examining and/or processing ofmedical data, e.g., as received as shown by box 210 and/or generated asdescribed herein. In cases where the imaging data is patient specific,e.g., is produced by imaging or scanning a real specific patient or isextended based on specific patient image data, the anatomical model maybe patient specific too, namely, the model generated as described hereinmay represent a specific, real patient. Model generation may beaccording to methods or other aspects as described in US PatentApplication Publication US 2009/0177454.

As shown by box 225, the method may include simultaneously performing asimulation of an image-guided procedure (that may be based on thegenerated anatomical model) and presenting related medical image data.For example, while the simulated procedure is in progress, relatedmedical image data may be displayed as described herein. In cases wherethe medical image data used for generating a model as described hereinis patient specific, the model generated as described herein may also bepatient specific and, accordingly, a simulation of a procedure asdescribed herein may be patient specific, namely, reflect, simulate orbe otherwise related to a real procedure performed on a real, specificpatient. According to embodiments of the invention, a simulatedprocedure may comprise a graphical representation of an anatomical modelthat may be displayed on a monitor with additional information, such assimulated models of tools. In some embodiments, the graphicalrepresentation of the anatomical structure or organ and of the tools mayexhibit real anatomical or physical qualities, traits, features, natureor aspects, e.g., move, bend, contract, react to pressure or medicine,bleed etc. A simulation of an image-guided procedure may comprise animage or graphical representation of an anatomical organ, e.g., a modelas described herein, that may be rotated or otherwise positioned, or maybe made to imitate a real anatomical system, e.g., change or evolve withtime, change shape in response to an operation of, or an interactionwith a medical tool or substance, bleed, or otherwise present or displayreal anatomical organ's behavior and related tools, medicine or otheraspects. For example, a catheter, stent or other tools, devices orelements may all be shown and further simulated. Accordingly, aphysician may perform a computerized simulation of the image-guidedprocedure (as shown by box 225) as a pre-procedure of the actual surgery(e.g., a surgical rehearsal or surgical simulation), part of a planningprocedure, as a training session or as a post-procedure.

As shown by box 225, the flow may include presenting related medicalimage data. According to embodiments of the invention, presentation ofrelated medical image data may be performed simultaneously orconcurrently with performance of a related simulation of an image-guidedprocedure, or it may be otherwise at the same time. In some embodiments,presentation of medical image data may be synchronized or otherwisecoordinated with a progress, state, mode, context or any relevant aspectof a simulated procedure. For example, in cases where the medical imagedata is related to a specific patient and, accordingly, the model andsimulation described herein may be patient specific as well, patientspecific medical image data, e.g., medical image data obtained from thespecific patient by an imaging system, may be presented together withthe simulated procedure, e.g., while the simulated procedure is inprogress. For example, a single display may be used to present medicalimage data and a simulated procedure at the same time, e.g., as shown inFIG. 3.

Reference is additionally made to FIG. 3 showing an exemplary screenshot300 related to an integrated presentation of a simulated medicalprocedure and an exemplary CT image data according to embodiments of theinvention. CT data is typically generated and stored as a stack ofimages, which may be referred to as slices forming together athree-dimensional (3D) box or volume of a scanned area. This volume maybe rendered as a whole using volume rendering techniques oralternatively a two-dimensional (2D) slice, also referred herein as a CTslice, may be sampled and displayed. For the sake of simplicity andclarity, reference herein will mostly be made to CT images or slices asknown in the art, however, it will be recognized that any medical imagedata, e.g., MRI, X-Ray, ultrasound or any suitable imaging or other datamay just as well be used and accordingly, embodiments of the inventionare not limited by the type, source or nature of the medical data.

As shown by FIG. 3, a display may be divided into a number of regions,for example, three regions 310, 320 and 330. A first region 310 may berelated to the simulated procedure and may accordingly present apatient-specific digital image-based model of an anatomical structure.In this exemplary display, the image-based model represents a vessel.Region 310 may further display a simulated or virtual medical tool modelrepresenting a physical intervention medical tool manipulated by a userduring a simulated procedure. For example, a catheter 312 may be shownwith relation to blood vessels 311. A simulated procedure may compriseshowing a movement of the catheter, e.g., as a result of actions takenby a physician, as well as reaction or other behavioral or other aspectsof the anatomical organs and tools. For example, as the simulationprogresses, catheter 312 may be shown to move through blood vessels 311,and blood vessels 311 may be shown to move, bend or otherwise react orexhibit phenomena typically seen or exhibited by real living organs.

As shown by 330 in FIG. 3, a second region or area may be dedicated orrelated to presenting related medical image data. For example and asshown by 332, a CT slice image may be shown. As described herein,various information, presentation modes and other features may beenabled with respect to the presentation of medical image data. Forexample, various elements shown in a simulation region 310 may berespectively shown, marked, highlighted, referenced or indicated in amedical image data region 330. Such correlation of views may be highlyappreciated by a person having ordinary skill in the art. For exampleand as shown by 331, a location of a simulated tool or part of asimulated tool that may be shown in the simulation region 310 may bemarked, highlighted, shown, referenced or indicated with respect to themedical image data. For example, the “+” indication shown by 331 mayrepresent the location of the tip of catheter 312 with respect to CTslice image 332 shown in region 330. Accordingly, a physician may beable to verify or determine the location of the tip of the catheter orotherwise orient himself by observing both the simulation region 310 andthe medical image data region 330.

In other embodiments, other aspects, details, parameters or relevantinformation may be indicated, shown, highlighted or referenced withrespect to the medical image data. For example, a specific blood vesselmay be shown or highlighted in the medical image data by a predefinedcolor thus enabling a user to easily determine the location of the bloodvessel with respect to the medical image data. Such blood vessel may besimilarly shown in the model thus enabling a user to correlate the viewprovided by the image-based model and the view provided by the medicalimage data. For example, a specific tissue, a specific organ or aspecific region that may be shown by the model may be highlighted,shown, referenced or otherwise indicated in or with reference to themedical image data.

As physicians and other medical personnel may be used to examininginformation by observing images such as CT images, integration of CT,MRI, X-Ray or other modalities with a simulated procedure as describedherein may have obvious benefits and may be highly appreciated by aperson having ordinary skill in the art.

Correlation, synchronization, coordination and/or integration of asimulated model and/or procedure with a presentation or related medicalimage data as described herein may be implemented and/or performedaccording to any applicable design or configuration. For example and asshown by FIG. 1, presentation unit 125 may interact with simulation unit115 in order to obtain any relevant information or parameters related toa state, context, progress or other aspect of a simulation. Presentationunit 125 may further interact with data repository 140 (or any otherapplicable data repository) in order to obtain medical image data.Accordingly, possessing any relevant information related to thesimulation and medical image data, presentation unit 125 may coordinate,synchronize or otherwise relate a presentation of medical image datawith the simulated procedure. For example, as the simulated model usedfor a simulated procedure may be generated based on medical image data,the simulated model may include references to related medical imagedata. As known in the art, cross references related to a number ofobjects, structures or elements may be used in order to relate elementsto one another. For example, a specific region or volume of a simulatedmodel may be based on specific CT frames or slices. Accordingly, crossreferences related to elements in a simulated model and image medicaldata elements may be maintained and used in order to correlate,coordinate, synchronize or otherwise relate a presentation of asimulated model and/or procedure with the relevant medical data. Forexample, presentation unit 125 may receive or obtain a reference to amedical image data element (e.g., a specific CT image) from simulationunit 115 or model generation unit 110 and may use such reference toretrieve the relevant CT image from data repository 140 and furtherdisplay such CT image as shown by 332.

As shown by 313, a medical image data element (e.g., a CT slice image)may be superimposed or overlaid on (or with) a presentation of asimulated model or alternatively, a simulated model may be superimposedon, or otherwise combined with, medical image data. Any presentationcombining an integrated presentation of medical image data elements suchas CT slice images and a simulated model as described herein may begenerated. For example and as shown by 313, a CT image slice may bedisplayed in the simulation region 310. A medical image data elementpresented in a simulation region as shown may be the same elementdisplayed in the dedicated medical image data region. For example, theCT slice image shown by 332 may be the same CT slice image shown by 313.Such display mode may further help a physician in the process byproviding an additional orientation aspect or tool as may readily beappreciated by a person having ordinary skill in the art. As shown, amedical image data element may be positioned or oriented in thesimulation region according to a specific view angle or other viewattributes or aspects. For example and as shown by 313 and 332, an axialslice view as known in the art may be presented. Other views may beenabled by embodiments of the invention, for example, based or accordingto user selection as described herein with respect to region 320. Asshown, a medical image data shown in conjunction with the simulatedmodel may enable a user to better interpret shown information, forexample, a user may quickly and easily determine the angle related tothe medical image, a location of anatomical parts and tools etc.

As shown by 320, a third region may comprise user interface elements,e.g., graphical user interface (GUI) objects, that may enable a user tocontrol various aspects of a simulated procedure and display of medicaldata as described herein. For example, a view of medical image datapresented in region 330 may be selected. For example and as shown, anaxial view (e.g., as shown by CT slice 332 in region 330), a sagittalview, a coronal view, a C-Arm view or a vessel selection view may beselected. Exemplary view selections or modes as shown in FIG. 3 are notto be construed as limiting embodiments of the invention, any otherapplicable view selections, modes or angles may be possible according toembodiments of the invention. Likewise, exemplary distribution ofgraphic elements and/or regions as shown by FIG. 3 are presented as anexample and may be different in different embodiments, for example, anumber of monitors or display screens may be used in order to displayinformation as described herein or any other arrangement of regions orgraphical elements may be possible.

Referring back to FIG. 2 and as shown by box 230, the flow may includecoordinating presentation of medical image data with a simulation of theimage-guided procedure. According to embodiments of the invention,various presentation modes may be selected. For example, presentation ofa set of medical data images may be automatic, e.g., according tovarious constraints, conditions, event or other aspects related to asimulated procedure. In some embodiments of the invention, medical imagedata may be automatically selected, presented, replaced or otherwisemanipulated, e.g., based on a progress, state and/or context of arelated to a simulated procedure. For example and as shown in region320, a checkbox may enable a user to select a mode of presenting medicalimage data according to a location of a tool or part of a tool. Forexample, checking the “follow tip” checkbox shown in region 320 maycause embodiments of the invention to update a presentation of medicalimage data based on the location of a tip of a simulated catheter, e.g.,the tip of catheter 312. For example, a CT slice may be selected andpresented based on the location of the tip of catheter 312 with respectto the blood vessels 311 shown in region 310. Accordingly, presentationof medical image data may be according to a progress of a simulatedprocedure.

Alternatively or additionally, medical image data may be presented basedon a selection of a location or region in the simulated model. Forexample, upon detecting a selection, e.g., a click of a mouse on aspecific location or element of a simulated model, the relevant medicalimage data may be displayed, e.g., in the dedicated region or area.Accordingly, a physician may be presented with a CT, MRI, X-Ray or othermodality's data with respect to a simulated model. Such presentation maybe performed during a progress of a simulated procedure or in offlinemode. For example, the physician may pause, stop or terminate asimulated procedure, examine the simulated model (that may be static orfrozen), and further select to review CT or MRI images (that may bereal, actual images taken from a specific patient as described herein)related to a specific location in the model, e.g., by clicking alocation of interest on the model. Presenting medical image data basedon a selection of a location on a simulated model may be accomplished byutilizing cross references as described herein.

As shown by box 235, the flow may include receiving a view selection.For example, a user may select one of the view selections shown inregion 320 in FIG. 3. As shown by box 240, the flow may includepresenting, while the simulated procedure is in progress, relatedmedical image data according to received view selection. For example,one of three orthogonal views as known in the art, axial (as shown byFIG. 3), coronal and sagittal may be selected. For example, a user mayselect the sagittal view by pressing the appropriate button shown inregion 320 and be provided with a presentation of a simulation andmedical image data according to a sagittal view as shown in FIG. 4.

Reference is additionally made to FIG. 4 showing an exemplary screenshot400 related to an integrated presentation of a simulated procedure andmedical image data according to embodiments of the invention. As shownby regions 410 and 430 in FIG. 4, a sagittal view as known in the artmay be provided, e.g., upon selection of such view as described herein.As known in the art, a sagittal or other view may be selected in orderto better determine aspects of anatomical elements or otherwise enable abetter or more suitable view, e.g., when other views do not fullyreveal, disclose or expose areas a physician may be interested in.Accordingly, when applicable, a physician or user of a systemimplementing embodiments of the invention may select a sagittal view andmay be provided with a presentation as shown in FIG. 4. Any relevantembodiments, features or aspects described herein, e.g., integration ofmedical image data with a simulated procedure, automatic selection,replacement or presentation of CT or other image slices, automaticpresentation of medical image data according to a progress of asimulation or superimposing of medical image data on a simulated modelmay be applicable to any of the views described herein, e.g., withreference to FIGS. 4, 5, 6, 7 and 8. For example, and as shown by 431, alocation of a tool may be indicated with respect to the medical imagedata, e.g., similarly to information indicated as described withreference to 331 in FIG. 3.

According to embodiments of the invention, in addition to the orthogonalviews described herein, non-orthogonal views may be enabled and/orpresented. Reference is additionally made to FIG. 5 showing an exemplaryscreenshot 500 related to an integrated presentation of a simulatedprocedure and medical image data according to embodiments of theinvention. As shown by FIG. 5, a vessel section view may be provided,e.g., upon selection of such view by a user as described herein.According to embodiments of the invention, a user may select a vesselsection view, for example, in order to “zoom in” on a specific vessel.In some embodiments, a relevant CT slice may be displayed alongnon-orthogonal ('oblique') sections as shown by 513. As shown, a CTslice placed according to a plane perpendicular to a vessel of interestmay be displayed in region 530 and may further be shown or representedin region 510 as shown by 513. Such CT slice may typically be positionedso that the vessel of interest is displayed as a circle in the center ofregion 530. Those having ordinary skill in the art may readilyappreciate the benefit of a vessel section view combined with asimulation as described herein, such integrated view may enable aphysician to easily examine a position of a tool within an organ, e.g.,a position of a catheter guided through a blood vessel. For example,such view may prove highly desirable in helping a physician navigate acatheter that may be blocked by obstacles within a blood vessel.

Any element of interest may be automatically located at a predefinedlocation in regions 630, 530, 430 and/or 330. For example, a medicalimage data, e.g., CT, may be shown such that a medical tool related tothe simulated model or the simulation is shown in a predefined location,e.g., in the center of region 330. Alternatively or additionally, aselected anatomical organ may be displayed in a predefined location. Forexample, a CT slice image may be positioned in regions 330, 430, 530 or630 such that a selected blood vessel is shown in the center of theseregions.

Another possible view that may be selected and provided according toembodiments of the invention may be a C-Arm view. As an arm typicallyused for positioning an X-Ray camera is typically in the shape of the“C” letter, the name C-Arm view may be appropriate for referring to aview as available to an X-Ray camera. According to embodiments of theinvention, upon selection of such view, a user may be provided with apoint of view of an X-Ray camera, namely, the user is provided with aview of the anatomical region as seen from the position of a relatedX-Ray camera or another imaging device placed at such position. Suchview may be well appreciated by physicians who are used to examiningX-Ray output or images. Reference is additionally made to FIG. 6 showingan exemplary screenshot 600 related to an integrated presentation of asimulated procedure and medical image data according to embodiments ofthe invention. As shown by 611, a position of an imaging device (thatmay typically be an X-Ray camera) may be shown, thus enabling aphysician to relate displayed information to the position of the imagingdevice at the time the image was acquired. As described herein, region630 in FIG. 6 may, similarly to regions 330, 430 and 530, presentsimulated X-Ray images. Region 610 in FIG. 6 may present a simulatedmodel, e.g., similarly to regions 310, 410 and 510. As described herein,the medical image data shown in region 610 in FIG. 6 may be superimposedon the simulated model in region 610 of FIG. 6 such that a user mayeasily realize the relation of the medical image data with the simulatedmodel.

Reference is additionally made to FIG. 7 showing an exemplary renderingmode 700 according to embodiments of the invention. According toembodiments of the invention and as shown in FIG. 7, a combinedrendering of Axial, Sagittal and Coronal slices (over a 3D anatomymodel) may be performed and displayed. For example, instead or inaddition to presenting a 2D slice in region 330 as shown in FIGS. 3through 6, any number of different slices can be jointly displayed overa single model as shown by FIG. 7. Such view may be desirable in variouscases, e.g., when a volume view is required or helpful. Such combinedslices view may be displayed in one of regions 330, 430, 530 and/or 630instead of or in addition to a 2D view, e.g., a CT slice image asdescribed herein. Yet another view that may be presented is shown inFIG. 8.

According to embodiments of the invention, volume rendering of medicalimage data may be used to produce a three dimensional view of themedical image data. Various rendering modes may be enabled and/orcombined with view selections described herein. For example, combinedslices rendering or volume rendering may be performed upon selection ofa user and may further be implemented in conjunction with selected viewssuch as those described herein.

Reference is additionally made to FIG. 8 showing an exemplary renderingmode 800 according to embodiments of the invention. As shown in FIG. 8,medical image data may be presented via or using volume rendering. Usingvolume rendering, images or other graphical information may bedisplayed, for example, in region 330 of FIGS. 3 through 6 and may behelpful when volume or depth aspects of an anatomical object or organneed to be evaluated or examined. A volume rendered image as shown inFIG. 8 may typically be associated with a transparency parameter and/orsetting such that its transparency may be controlled in order to enableviewing parts or sections that may otherwise be hidden, e.g., by partsor regions containing or otherwise concealing them. Accordingly,variable transparency may be enabled. Referring back to FIG. 2, as shownby box 245, the flow may include storing related information. Forexample, a simulated procedure, including any view selection or movementof tools may be recorded and such recording may be stored and laterused, e.g., in a training session.

Embodiments of the invention may include an article such as a computeror processor readable medium, or a computer or processor storage medium,such as for example a memory, a disk drive, or a USB flash memory,encoding, including or storing instructions, e.g., computer-executableinstructions, which when executed by a processor or controller, carryout methods disclosed herein.

Although embodiments of the invention are not limited in this regard,the terms “plurality” and “a plurality” as used herein may include, forexample, “multiple” or “two or more”. The terms “plurality” or “aplurality” may be used throughout the specification to describe two ormore components, devices, elements, units, parameters, or the like.

Unless explicitly stated, the method embodiments described herein arenot constrained to a particular order or sequence. Additionally, some ofthe described method embodiments or elements thereof can occur or beperformed at the same point in time or overlapping points in time. Asknown in the art, an execution of an executable code segment such as afunction, task, sub-task or program may be referred to as execution ofthe function, program or other component.

Although embodiments of the invention are not limited in this regard,discussions utilizing terms such as, for example, “processing,”“computing,” “calculating,” “determining,” “establishing”, “analyzing”,“checking”, or the like, may refer to operation(s) and/or process(es) ofa computer, a computing platform, a computing system, or otherelectronic computing device, that manipulate and/or transform datarepresented as physical (e.g., electronic) quantities within thecomputer's registers and/or memories into other data similarlyrepresented as physical quantities within the computer's registersand/or memories or other information storage medium that may storeinstructions to perform operations and/or processes.

While certain features of the invention have been illustrated anddescribed herein, many modifications, substitutions, changes, andequivalents may occur to those skilled in the art. It is, therefore, tobe understood that the appended claims are intended to cover all suchmodifications and changes as fall within the true spirit of theinvention.

1. A method of performing computerized simulations of image-guided procedures, the method comprising: receiving medical image data related to a specific patient; producing a patient-specific digital image-based model of an anatomical structure of the specific patient based on the medical image data; performing a computerized simulation of an image-guided procedure; and simultaneously displaying the image-based model and at least some of the medical image data of the specific patient.
 2. The method of claim 1, wherein displaying comprises overlaying at least some of the medical image data and the image-based model to form an integrated display.
 3. The method of claim 1, wherein the anatomical structure is a vessel and the medical image data is displayed in a plane perpendicular to the vessel.
 4. The method of claim 1, wherein the simulation comprises a simulated model of a physical medical tool and the simulation comprises manipulating the simulated model of the medical tool.
 5. The method of claim 3, wherein a representation of the medical image data is integrated with the image-based model such that the plane is positioned at a location corresponding to a location of a simulated medical tool.
 6. The method of claim 4, wherein displaying comprises displaying on a specific location within a representation of the medical image data an indication representing the simulated medical tool, the specific location corresponding to a location of at least a part of the simulated model of the medical tool in relation to the image-based model.
 7. The method of claim 4 comprising: synchronizing a display of the medical image data with movement of the simulated model of the medical tool.
 8. The method of claim 1, wherein the medical image data is received form a picture archiving and communication systems (PACS) archive.
 9. The method of claim 1, wherein the medical image data is received from one of: a computerized tomography (CT) system, a magnetic resonance imaging (MRI) system, a X-Ray system, a positron emission tomography (PET) system, an Ultrasound system and a fluoroscopy system.
 10. The method of claim 1, comprising generating artificial medical image data based on received raw medical image data and simultaneously presenting the image-based model and at least some of the artificial medical image data.
 11. The method of claim 1, comprising generating, based on the received raw medical image data, an image related to a specific plane and displaying the image to provide a specific plane view.
 12. The method of claim 11, comprising overlaying said image on said image-based model to produce an integrated display of said image-based model and said specific plane view.
 13. The method of claim 11, wherein said specific surface is related to a plane and wherein said plane is one of: an axial plane, a coronal plane, a sagittal plane, a plane orthogonal to a selected vessel, a non-planar surface and a plane related to an imaging system used to acquire said medical image data.
 14. The method of claim 11, wherein said image is rendered to provide a specific surface view using maximum intensity projection (MIP).
 15. The method of claim 1, wherein displaying the medical image data comprises a volume rendering of said medical image data to produce a three dimensional (3D) view of said medical image data.
 16. The method of claim 1, wherein displaying the medical image data comprises least one of: highlighting, marking, showing, referencing and indicating at least one of: a tissue, an organ and a region.
 17. An article comprising a computer-storage medium having stored thereon instructions that, when executed by a processing platform, result in: receiving medical image data of a specific patient; producing a patient-specific digital image-based model of an anatomical structure of the specific patient based on the medical image data; performing a computerized simulation of an image-guided procedure; and simultaneously displaying the image-based model and at least some of the medical image data of the specific patient.
 18. The article of claim 17, wherein the instructions when executed further result in overlaying the medical image data and the image-based model to form an integrated display.
 19. The article of claim 17, wherein the anatomical structure is a vessel and the medical image data is displayed in a plane perpendicular to the vessel.
 20. The article of claim 17, wherein the simulation comprises a simulated model of a physical medical and the simulation comprises manipulating the medical tool.
 21. The article of claim 20, wherein a representation of the medical image data is integrated with the image-based model such that the plane is positioned at a location corresponding to a location of the simulated medical tool.
 22. The article of claim 20, wherein displaying comprises displaying on a specific location within a representation of the medical image data an indication representing the simulated medical tool, the specific location corresponding to a location of at least a part of the simulated model of the medical tool in relation to the image-based model.
 23. The article of claim 20, wherein the instructions when executed result in synchronizing a display of the medical image data with movement of the simulated model of the medical tool.
 24. The article of claim 17, wherein the medical image data is received form a picture archiving and communication systems (PACS) archive.
 25. The article of claim 17, wherein the medical image data is received from one of: a computerized tomography (CT) system, a magnetic resonance imaging (MRI) system, a X-Ray system, a positron emission tomography (PET) system, an Ultrasound system and a fluoroscopy system.
 26. The article of claim 17, wherein the instructions when executed result in generating artificial medical image data based on received raw medical image data and simultaneously presenting the image-based model and at least some of the artificial medical image data.
 27. The article of claim 17, wherein the instructions when executed result in generating, based on the received raw medical image data, an image related to a specific plane and displaying the image to provide a specific plane view.
 28. The article of claim 17, wherein the instructions when executed further result in overlaying said image on said image-based model to produce an integrated display of said image-based model and said specific plane view.
 29. The article of claim 27, wherein said specific plane is related to a plane and wherein said plane is one of: an axial plane, a coronal plane, a sagittal plane, a plane orthogonal to a selected vessel, a non-planar surface and a plane related to an imaging system used to acquire said medical image data.
 30. The article of claim 17, wherein the said image is rendered to provide a specific surface view using maximum intensity projection (MIP).
 31. The article of claim 17, wherein displaying the medical image data comprises a volume rendering of said medical image data to produce a three dimensional (3D) view of said medical image data.
 32. The article of claim 17, wherein displaying the medical image data comprises at least one of: highlighting, marking, showing, referencing and indicating at least one of: a tissue, an organ and a region.
 33. A system for performing computerized simulations of image-guided procedures, the system comprising: an imaging unit to generate medical image data of a patient; a model generation unit to receive the medical image data and to produce a patient-specific digital image-based model of an anatomical structure of the specific patient based on the medical image data; a simulation unit to perform a computerized simulation of an image-guided procedure by comprising manipulating a physical medical tool; and a presentation unit to cause a simultaneous display of the digital image-based model, a simulated model of the medical tool and at least part of the medical image data. 